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Access multimedia info to be displayed in the GU I 
1 1402 



Extract text info from the multimedia info accessed in step 1402 
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Store the text info extracted in step 1404 
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Determine a length or height (in pixels) of a text canvas for drawing the 
text information 



Calculate multipliers (pix_m, secjri) for converting pixels locations in the 
text canvas to seconds and for converting seconds to pixels locations in 
the text canvas 



Calculate positional coordinates (horizontal (X) and vertical (Y) 
coordinates) for words in the text information extracted in step 1404 
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Draw the words in the text information on the text canvas in a location 
determined by the X and Y coordinates calculated for the words in step 
1412 
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Determine height of thumbnail that displays text information in the 
second viewing area of the GUI 



Generate a thumbnail by scaling the text canvas such that the height of 
thumbnail is equal to height determined in step 1416 and the thumbnail 
fits entirely within the size constraints of the second viewing area 
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Display the thumbnail generated in step 1418 in the second viewing 
area of the GUI 
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Calculate multipliers (fp/x_m, fsec_m) for converting pixels locations in 
the text thumbnail to seconds and for converting seconds to pixels 
locations in the text thumbnail 
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Extract a set of keyframes from the video information included in the 
multimedia information 
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Video keyframes extracted in step 1502 and their associated timestamp 
information is stored in a data structure 
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Video keyframes extracted in step 1504 are divided into groups 



Calculate a start time and an end time for each group of frames 
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For each group of keyframes, determine a segment of pixels on a 
keyframe canvas for drawing one or more keyframes from the group of 
keyframes 
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Determine a number of keyframes from each group of frames to be 
drawn in each segment of pixels on the text canvas 



For each group of keyframes, identify one or more keyframes from the 
keyframes in the group of keyframes to be drawn on the text canvas 
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For each group of keyframes, draw keyframes from the group identified 
in step 1514 on the keyframe canvas in the segment of pixels 
corresponding to the group 



Determine a height (or length) of a thumbnail that displays the video 
keyframes 
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Generate the thumbnail by scaling the keyframe canvas such that the 
height of the thumbnail is equal to the height determined in step 1518 
and the thumbnail fits entirely within the size constraints of the second 
viewing area 
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Display the thumbnail generated in step 1520 in the second viewing 
area of the GUI 
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Calculate multipliers 
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Extract a set of keyframes from the video information included in the 
multimedia information 
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Video keyframes extracted in step 1602 and their associated timestamp 
information is stored in a data structure 
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Video keyframes extracted in step 1604 are divided into groups 
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Calculate a start time and an end time for each group of frames 
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Select one or more groups of video keyframes from the groups 
determined in step 1606 
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For each group selected in step 1609, identify one or more video 
keyframes from the group to be drawn on the keyframe canvas 
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Divide the keyframe canvas into a number of equal-sized row portions, 
where the number of row portions is equal to the number of groups 
selected in step 1609 
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For each group selected in step 1609, identify a row portion on the 
keyframe canvas for drawing video keyframes from the group 
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For each group selected in step 1609, draw one or more video 
keyframes (identified in step 1610) from the group in a row portion of the 
keyframe canvas identified in step 1614 
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Determine a height (or length) of a thumbnail that displays the video 
keyframes 
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Generate the thumbnail by scaling the keyframe canvas such that the 
height of the thumbnail is equal to the height determined in step 1618 
and the thumbnail fits entirely within the size constraints of the second 
viewing area 



Display the thumbnail generated in step 1620 in the second viewing 
area of the GUI 
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Determine a height (in pixels) of panels to be displayed in third viewing 
area of the GUI 
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Identify a section of the text canvas equal to the height determined in step 
1702 
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Determine time values corresponding to the boundaries of the section of 
the text canvas identified in step 1704 (marked by pixel locations P sfeJrt and 
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Identify a section of the keyframe canvas corresponding to the selected 
section of the text canvas 



Display the portion of the text canvas identified in step 1704 in a panel in 
the third viewing area of the GUI 



Display the portion of the keyframe canvas identified in step 1708 in a 
panel in the third viewing area of the GUI 
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Display a panel viewing area lens covering a section of the third viewing 
area in the GUI 
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Determine a portion of thumbnail 312-1 corresponding to the section of text 

canvas displayed in panel 324-1 and a portion of thumbnail 312-2 
corresponding to the section of keyframe canvas displayed in panel 324-2 
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Display thumbnail viewing area lens covering portions of thumbnails 312-1 
and 312-2 corresponding to the section of text canvas displayed in panel 
324-1 and the section of keyframe canvas displayed in panel 324-2 



Determine a portion of second viewing area corresponding to the section of 
third viewing area emphasized by the panel viewing area lens 
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Display a sub-lens covering portions of thumbnails 312-1 and 312-2 
corresponding to the portion of the third viewing area emphasized by the 
panel viewing area lens 322 
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Display multimedia info corresponding to the portion of third viewing area 
emphasized by the panel viewing area lens in the fourth viewing area 



Analyze the multimedia information and display results of the analysis in 
the fifth viewing area of the GUI 
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Detect a change in the position of the thumbnail viewing area lens from 
a first position to a second position over the second viewing area 



Determine a portion of the second viewing area emphasized by the 
thumbnail viewing area lens in the second position 
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Determine time values corresponding to the second position of the 
thumbnail viewing area lens 
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Determine pixel locations in the text canvas and the keyframe canvas 
corresponding to the time values determined in step 1806 



Display a section of the text canvas between pixel locations P sfart and 
P end m panel 324-1 



Display a section of the keyframe canvas between pixel locations P sfe 
and P end in panel 324-2 
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Determine a portion of the second viewing area emphasized by the sub- 
lens in the second position 
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Determine time values corresponding to the second position of the sub- 
lens 
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Determine pixel locations in the text canvas and the keyframe canvas 
corresponding to the time values determined in step 1816 
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Panel viewing area lens is drawn over third viewing area covering a 
portion of the third viewing area between pixels locations determined in 
step 1818 
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Display multimedia info in the fourth viewing area corresponding to the 
new position of the panel viewing area lens 
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Detect a change in the position of the panel viewing area lens from a 
first position to a second position over the third viewing area 
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Determines time values (t 3 , t 4 ) corresponding to the second position of 
the panel viewing area lens 



Determine pixel locations in the second viewing area corresponding to 
the time values determined in step 1904 



Update position of sub-lens 31 6 to emphasize a portion of thumbnails 
312 in the second viewing area between the pixel locations determined 
in step 1906 



Display multimedia info corresponding to the second position of the 
panel viewing area lens in the fourth viewing area 
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Create one or more ranges based upon the locations 
of the hits identified in step 2104 
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Display the one or more ranges created in step 2106 
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Determine a time associated the first hit in the 
multimedia info 
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Create new range for the next hit with R s of the new 
range set to time determined in step 2208 



Extend the range including the previous hit by 

changing the end time (R E ) of the range to the time 

determined in step 2208 
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